System and method for detecting trachea

ABSTRACT

Disclosed are systems, devices, and methods for detecting a trachea, an exemplary system comprising an imaging device configured to obtain image data and a computing device configured to generate a three-dimensional (3D) model, identify a potential connected component in a first slice image, identify a potential connected component in a second slice image, label the first slice image as a top slice image, label the connected component in the top slice image as an active object, associate each connected component in a current slice image with a corresponding connected component in a previous slice image based on a connectivity criterion, label each connected component in the current slice image associated with a connected component of the preceding slice image as the active object, and identify the active object as the trachea, based on a length of the active object.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of and priority to U.S.Provisional Patent Application Ser. No. 62/020,257 filed on Jul. 2,2014, the entire contents of which are incorporated herein by reference.

BACKGROUND Technical Field

The present disclosure relates to systems and methods for detecting atrachea. More particularly, the present disclosure relates to systemsand methods that detect a trachea based on slice images of athree-dimensional volume of a chest.

Discussion of Related Art

Visualization techniques related to visualizing a chest have beendeveloped so as to help clinicians perform diagnoses and/or surgeries onorgans or other parts contained within the chest. Visualization isespecially important for identifying a location of a diseased region.Further, when treating the diseased region, additional emphasis is givento identification of the particular location of the diseased region sothat a surgical operation is performed at the correct location in thechest.

In the past, scanned two-dimensional images of the chest have been usedto aid in visualization. In order to visualize a lung from scannedtwo-dimensional images of the chest, it is important to determinewhether or not an area of the two-dimensional images is a part of thelung. Thus, detecting a starting location, for example, a location of anorgan or other part that is connected to or is a part of the lung, isalso important for identifying the lung. In one example, the trachea canbe used as the starting location because the trachea has a substantiallyconstant diameter along its length and is known to be connected to thelung.

SUMMARY

Provided in accordance with the present disclosure is a system fordetecting a trachea of a patient.

In an aspect of the present disclosure, the system includes an imagingdevice configured to obtain image data of the patient, and a computingdevice including a processor and a memory storing instructions which,when executed by the processor, cause the computing device to generate athree-dimensional (3D) model of a chest of the patient based on theimage data, generate slice images of the 3D model along an axialdirection, identify a potential connected component in a first sliceimage of the generated slice images, identify a potential connectedcomponent in a second slice image of the generated slice images, whereinthe second slice image is immediately subsequent to the first generatedslice image, confirm that the potential connected component of the firstand second slice images are connected, label the potential connectedcomponent as a connected component, label the first slice image as a topslice image of the generated slice images, label the connected componentin the top slice image as an active object, associate each connectedcomponent in a current slice image of the generated slice images with acorresponding connected component in a previous slice image based on aconnectivity criterion, label each connected component in the currentslice image associated with a connected component of the preceding sliceimage as the active object, and identify the active object as thetrachea, based on a length of the active object.

In another aspect of the present disclosure, the image data is obtainedby an imaging device using a tomographic technique, radiography,tomogram produced by a computerized axial tomography scan, magneticresonance imaging, ultrasonography, contrast imaging, fluoroscopy,nuclear scans, or positron emission tomography.

In a further aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image.

In another aspect of the present disclosure, the slice images are spacedat an equal distance apart from each other.

In a further aspect of the present disclosure, the instructions furthercause the computing device to calculate a length of a finalized activeobject by multiplying a number of slice images contained in thefinalized active object minus one and the distance between each sliceimage.

In another aspect of the present disclosure, when the length of thefinalized active object is greater than or equal to 70 mm, theinstructions further cause the computing device to indicate that thetrachea is identified.

In a further aspect of the present disclosure, when the length of thefinalized active object is greater than or equal to 30 mm but less than70 mm, the instructions further cause the computing device to indicatethat the trachea is potentially identified.

In another aspect of the present disclosure, when the length of thefinalized active object is less than 30 mm, the instructions furthercause the computing device to indicate that the trachea is notidentified.

In a further aspect of the present disclosure, a connected component ofthe current slice image is associated with the corresponding connectedcomponent in the previous slice image when coordinates of a pixel in theconnected component of the current slice image matches coordinates of apixel in the corresponding connected component in the previous sliceimage.

In another aspect of the present disclosure, a connected component ofthe current slice image is associated with the corresponding connectedcomponent in the previous slice image when a difference between a centerof mass of the connected component of the current slice image and acenter of mass of the corresponding connected component in the previousslice image is less than a predetermined value.

In a further aspect of the present disclosure, a connected component ofthe current slice image is associated with a corresponding connectedcomponent in the previous slice image when a difference between an areaof the connected component of the current slice image and an area of thecorresponding connected component in the previous slice image is lessthan a predetermined value.

In another aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image based on an association parameter, and wherein theassociation parameter is an area ratio calculated by dividing an area ofthe connected component in the current slice image by an area of thecorresponding connected component in the previous slice image.

In a further aspect of the present disclosure, wherein the instructionsfurther cause the computing device to finalize the active object in theprevious slice image based on an association parameter, and wherein theassociation parameter is a ratio between a number of coordinates of theconnected component of the current slice image, which match coordinatesof the corresponding active object in the previous slice image, and anumber of non-matching coordinates of the connected component of thecurrent slice image

In another aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image based on an association parameter, and wherein theassociation parameter is an area of the connected component of thecurrent slice image.

In a further aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image based on an association parameter, and remove the label ofthe corresponding active object of the previous slice as an activeobject when the association parameter is greater than a predeterminedvalue.

In another aspect of the present disclosure, the instructions furthercause the computing device to remove the label of the connectedcomponent of the current slice as an active object when the associationparameter is greater than the predetermined value.

In a further aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image based on an association parameter, and wherein an activeobject is finalized when the association parameter is less than apredetermined value.

In another aspect of the present disclosure, the instructions furthercause the computing device to finalize the active object in the previousslice image based on an association parameter, and label the connectedcomponent of the current slice as the active object when the associationparameter is greater than or equal to a first predetermined value andless than or equal to a second predetermined value.

Any of the above aspects and embodiments of the present disclosure maybe combined without departing from the scope of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Objects and features of the presently disclosed systems and methods willbecome apparent to those of ordinary skill in the art when descriptionsof various embodiments are read with reference to the accompanyingdrawings, of which:

FIG. 1 is a schematic diagram of an example device which may be used todetect a trachea in a 3D model of a patient's lungs, in accordance withan embodiment of the present disclosure;

FIG. 2 depicts 2D slice images generated from the 3D model showing thetrachea in the axial and coronal orientations, in accordance withembodiments of the present disclosure;

FIG. 3 is a graphical illustration of connected components in 2D sliceimages of a a patient's chest in accordance with embodiments of thepresent disclosure;

FIG. 4 is a graphical illustration of a planar view of 2D slice imagesof the patient's chest in accordance with embodiments of the presentdisclosure;

FIG. 5A is a flowchart of a method for detecting a trachea in accordancewith embodiments of the present disclosure; and

FIG. 5B is a flowchart of a method for determining an associationbetween 2D slice images in accordance with embodiments of the presentdisclosure.

DETAILED DESCRIPTION

The present disclosure is related to systems and methods forautomatically detecting a trachea based on 2D slice images of apatient's chest. Identifying the trachea may be a necessary component ofpathway planning for performing an ELECTROMAGNETIC NAVIGATIONBRONCHOSCOPY® (ENB) procedure using an electromagnetic navigation (EMN)system.

An ENB procedure generally involves at least two phases: (1) planning apathway to a target located within, or adjacent to, the patient's lungs;and (2) navigating a probe to the target along the planned pathway.These phases are generally referred to as (1) “planning” and (2)“navigation.” By detecting the trachea, the lung can be visuallydistinguished from areas outside of the lung because the lung isconnected to the trachea. An example of the planning software describedherein can be found in U.S. patent application Ser. Nos. 13/838,805,13/838,997, and 13/839,224, all of which were filed by Covidien LP onMar. 15, 2013, and entitled “Pathway Planning System and Method,” all ofwhich are incorporated herein by reference. An example of the planningsoftware can be found in commonly assigned U.S. Provision PatentApplication No. 62/020,240 entitled “SYSTEM AND METHOD FOR NAVIGATINGWITHIN THE LUNG” the entire contents of which are incorporated herein byreference.

Prior to the planning phase, the patient's lungs are imaged by, forexample, a computed tomography (CT) scan, although additional applicablemethods of imaging will be known to those skilled in the art. The imagedata assembled during the CT scan may then be stored in, for example,the Digital Imaging and Communications in Medicine (DICOM) format,although additional applicable formats will be known to those skilled inthe art. The CT scan image data may then be loaded into a planningsoftware application (“application”) to be processed for generating a 3Dmodel which may be used during the planning phase of the ENB procedure.

The application may use the CT scan image data to generate a 3D model ofthe patient's lungs. The 3D model may include, among other things, amodel airway tree corresponding to the actual airways of the patient'slungs, and showing the various passages, branches, and bifurcations ofthe patient's actual airway tree. While the CT scan image data may havegaps, omissions, and/or other imperfections included in the image data,the 3D model is a smooth representation of the patient's airways, withany such gaps, omissions, and/or imperfections in the CT scan image datafilled in or corrected.

The planning phase generally involves identifying at least one target inthe 3D model, and generating a pathway to the target. The pathway willgenerally run from the patient's mouth, through the trachea andconnected airways, to the target. However, in order to generate thepathway to the target, the location of the trachea within the 3D modelmust be known.

As described in more detail below, the application will attempt toautomatically detect the trachea within the 3D model. However, there maybe instances where automatic detection of the trachea fails. In suchinstances, the trachea may need to be manually identified and marked.This process is more fully described in commonly-owned U.S. ProvisionalPatent Application Ser. No. 62/020,253 entitled “Trachea Marking”, filedon Jul. 2, 2014, by Lachmanovich et al., the entire contents of whichare hereby incorporated by reference.

The trachea provides a passage way for breathing. The trachea isconnected to the larynx and the pharynx in the upper end. In particular,the upper part of the trachea extends substantially linearly from thelarynx and pharynx and behind the sternum. The lower end of the tracheabranches into a pair of smaller tubes, i.e., primary bronchi, each tubeconnecting to a lung. The main carina is a cartilaginous ridge formed bythe branching of the trachea into the primary bronchi. The diameter ofthe trachea is substantially constant along its length (i.e., the axialdirection), while the size of the lung changes substantially along thesame direction as the length of the trachea. Thus, by analyzing 2D sliceimages of the 3D model, the trachea may be detected.

FIG. 1 shows an image processing device 100 that may be used during theplanning phase of an ENB procedure to detect the location of the tracheain the 3D model. Device 100 may be a specialized image processingcomputer configured to perform the functions described below. Device 100may be embodied in any form factor known to those skilled in the art,such as, a laptop, desktop, tablet, or other similar computer. Device100 may include, among other things, one or more processors 110, memory120 storing, among other things, the above-referenced application 122, adisplay 130, one or more specialized graphics processors 140, a networkinterface 150, and one or more input interfaces 160. As noted above, 2Dslice images of the 3D model may be displayed in various orientations.As an example, FIG. 2 shows 2D slice images of the 3D model of thepatient's lungs in the axial and coronal orientations, with 2D sliceimage 210 generated along the axial plane and 2D slice image 220generated along the coronal plane.

Both 2D slice images 210 and 220 show the trachea 212 and the maincarina 214. The 2D slice images of the 3D model may show a high densityarea with high intensity and a low density area with low intensity. Forexample, bones, muscles, blood vessels, or cancerous portions aredisplayed with higher intensity than an inside area of airways of thelung.

In an aspect, the 2D slice images may be generated to depict the axial,coronal, and sagittal views of the patient at a given location. Forexample, at each intersecting point of the 3D model, there may be threedifferent 2D slice images generated in the three independent directions.These 2D slice images may be reformatted for display. For example,application 122 may convert a color space of the 2D slice images toanother color space suitable for display and perform imaging processes,e.g., scale, rotation, translation, or projection, to display the 2Dslice images as intended.

The 2D slice images may be binarized by using a region growingalgorithm. Based on the region growing algorithm and starting with aseed pixel, every pixel in the 2D slice images of the 3D model ischecked to determine whether a Hounsfield value assigned to each pixelis less than a threshold value and whether each pixel is connected tothe seed pixel. When it is determined that a value assigned to a pixelhas a Hounsfield value less than the threshold value and is connected tothe seed pixel, the Hounsfield value of the pixel is set to one or themaximum value. Otherwise, the Hounsfield value of the pixel is set tozero or the minimum value. As part of the region growing algorithm, thethreshold is selected with a high enough value to cause leakage in thelung, and thus fill the lungs with intensity values leaked from theairways.

After every pixel in the 2D slice images of the 3D model is set to themaximum or minimum value, the 2D slice images will have only 2 colors ofpixels. The result is a set of 2D slice images where the pixels havingthe maximum Hounsfield value would appear white, and the pixels havingthe minimum Hounsfield value would appear black. In some instances, thevalues of pixels in the 2D slice images of the 3D model are inversed sothat the lung regions are shown in black and the non-lung regions areshown in white or another color. The binarized 2D slice images may showwhite regions as non-lung areas (e.g., bones, stomach, heart, bloodvessels, walls of airways, etc.) and black regions as lung areas (e.g.,the lung, the trachea, and connected components). As described in moredetail below, connected components are areas of a 2D slice image whichare identified as having corresponding areas in one or more of the other2D slice images, and thus may represent the patient's lungs or trachea.

FIG. 3 illustrates three 2D slice images generated based on the 3D modelin accordance with an embodiment of the present disclosure. Image 305 isgenerated along the axial direction, image 310 is generated along thesagittal direction, and image 315 is generated along the coronaldirection. Black areas shown in the three images 305, 310, and 315 arelung regions, and white areas included in the three images 305, 310, and315 are non-lung areas. The white areas may represent blood vessels andwalls of airways. In a case where an interior area of connectedcomponents is sufficiently large and has a lower density (e.g., blood,air, or coarse space) than tissue making up the lung regions, a blackarea also appears. In this sense, the connected components include alung area as well. For example, connected components in the image 305are the left lung 320, the right lung 325, the left primary bronchus330, and the right primary bronchus 335. White areas inside the leftlung 320 and the right lung 325 are not connected components but areblood vessels or walls of airways.

The upper part of the trachea extends substantially linearly from thelarynx and pharynx and behind the sternum or breastbone. The lower endof the trachea branches into a pair of smaller tubes, i.e., primarybronchi, each tube connecting to a lung. The diameter of the trachea issubstantially constant along its length (i.e., the axial direction),while the size of the lung changes substantially along the samedirection as the length of the trachea. Thus, by analyzing areas ofconnected components in each 2D slice image generated based on the 3Dmodel, the trachea may be detected. For this reason, images generatedalong the axial direction may be analyzed to detect the trachea in thispresent disclosure. In other embodiments, images generated along theother two directions may also be used to detect the trachea.

FIG. 4 shows 2D slice images generated from the 3D model in accordancewith embodiments of the present disclosure. Image 405 is a coronal imageof the patient depicting the axial locations along the patient at whichaxial images 410 a-430 b are identified and processed in accordance withthe present disclosure. For example, image 410 a is taken from an axialposition along the chest indicated by the top gray line, image 415 a istaken from another axial position along the chest indicated by thesecond gray line, image 420 a is taken from another axial position alongthe chest indicated by the third gray line, etc.

The axial locations of the images 410 a-430 b may be spaced an equaldistance from each other, meaning that a distance between any twoneighboring 2D slice images is the same distance D. The axial 2D sliceimages 410 a, 415 a, 420 a, 425 a, and 430 a depict a portion of thechest of the patient at different locations. As a result of thebinarization, each of these images 410 a, 415 a, 420 a, 425 a, and 430 ashow black enclosed areas which represent the trachea and or the lungtissue.

A process for detecting the trachea may be based on the identifiedconnected components in each axial 2D slice image 410 a, 415 a, 420 a,425 a, and 430 a. Generally, a first axial 2D slice image is analyzed toidentify one or more identified areas which satisfy the binarizationcriteria (i.e., are likely either trachea or lung). In addition toidentifying areas of the axial 2D slice image which satisfy thebinarization criteria, an initial connected component analysis is donewhich filters out any portion of the axial 2D slice image 410 a thatconnects to the picture borders. Further, connected components which areabove or below a certain size threshold are also filtered out. Theremaining connected components of any one axial image slice, e.g. 410 a,is associated with an active object depending on a connectivity criteriawith the connected components in other images. An axial connectedcomponent analysis is undertaken in which a determination is made as towhether connected components in two successive axial 2D slice imagesgeographically overlap with one another. Geographical overlap can bedetermined by comparing coordinates of the active object in thesuccessive images and determining if the same coordinates (e.g. X and Ycoordinates) appear in the active objects of successive images. If so,the connected components from the two axial 2D slice images areassociated with each other and are both correspondingly labeled as anactive object. A connected component labeled as the active object is acandidate to be identified as the trachea. When the additional connectedcomponents do not geographically overlap with the one or more connectedcomponents from the previous 2D slice image, the additional connectedcomponents are labeled as a new active object. Further, if in asubsequent axial slice it is determined that there are no connectedcomponents objects which overlap with the preceding image, the activeobject last identified in the preceding image is finalized. Theabove-described steps are performed on each 2D slice image until eachconnected component in each coronal 2D slice image is identified and,where appropriate, classified as an active object.

The details of the process described above are further clarified withreference to FIG. 4. In an embodiment, the top axial 2D slice image 410a is processed first to identify or label the connected component 411.In one embodiment, any connected component in the top axial 2D sliceimage 410 a is labeled as an active object. As a result, in image 410 bof the filtering described above, a single active object 412 is shown.

Next, the second axial 2D slice image 415 a is processed in a similarmanner as coronal 2D slice image 410 a to identify three connectedcomponents 416, 417, and 418. Again, the filtering described above isundertaken, resulting in the identification of three active objects 416,417, and 418 depicted in image 415 b. A determination is made as towhether one or more of the connected components 416-418 geographicallyoverlap with connected components (e.g., 411) in the previous axial 2Dslice image. As a result of this analysis, active objects 413 and 414are new active objects, with no connected component to compare with inthe preceding axial 2D slice image 410 b. However, connected component416 geographically overlaps with and is associated with the connectedcomponent 411 in the 2D slice image 410 a, thus, connecting the twoconnected components 416 and 411 vertically (i.e. from axial slice toaxial slice) to each other. As a result the associated connectedcomponents 416 and 411 share a common active object label 412

With reference to a third axial 2D slice image 420 a, three connectedcomponents 421-423 are identified. Following the filtering describedabove, each connected component 421-423 is separately compared with theconnected components 416-418 of the second axial 2D slice image 415 a.The connected component 421 geographically overlaps with the connectedcomponent 416, and has a similar size or area with that of the connectedcomponent 416. Thus, the connected component 421 is associated with theconnected component 416 and labeled as the same active object 412 as theconnected component 416, which was based on its comparison to connectedcomponent 411 in axial image slice 410 a.

The connected components 422 and 423 geographically overlap with theconnected components 417 and 418, respectively and are thus candidatesto be labeled as active objects 413 and 414 based on this overlap. Theconnected components 422 and 423, however, must also be filtered bysize, as described above. Because the areas of the connected components422 and 423 are larger than a predetermined maximum size they must befiltered out of consideration as an active object. In FIG. 420 b, theseconnected components are shown as filtered out based on the change ofcolor from black to white. In contrast, connected component 421, whichis associated with active object 412 remains black. In the context ofthe present disclosure, because the trachea is known to have asubstantially consistent diameter along its length, and because thatdiameter is generally within a well known range of between about 27 and13 mm for men, and between about 23-10 mm in women, when a connectedcomponent is identified as having a substantially larger area than anarea of the corresponding connected component in the previous 2D sliceimage, an organ represented by such connected component is determined tobe something other than the trachea and thus excluded from the analysis.As an alternative or additional step, because the connected components422 and 423 have areas that are larger than those of connectedcomponents 416 and 418, the connected components 422 and 423 may also beconsidered too large and thus not part of the trachea. Further, theconnected components 417 and 418 of the second axial 2D slice image 415b may be re-labeled to remove the active object designation.Consequently, the 2D slice images 410 b, 415 b and 420 b have only oneactive object 412.

As described briefly above, a connected component of separate 2D sliceimages may be associated with a connected component of an adjacent upper2D slice image based on connectivity criteria. The connectivity criteriamay include consideration of equality of coordinates on the current 2Dslice image with coordinates of the adjacent upper 2D slice image. In anembodiment, the coordinates of a pixel of a 2D slice image may be basedon the Cartesian coordinate system, where the origin may be located inan upper left corner of the 2D slice image and coordinates increase fromleft to the right and from top to bottom. Alternatively, the coordinatesof a pixel may be based on another coordinate system, such as polarcoordinate system, which is suitable for intended purposes.

The geometric overlap between two connected components, also called anassociation parameter, from two different images may be calculated maybe based on the number of pixels of a connected component of the current2D slice image which match coordinates of pixels of a connectedcomponent of the adjacent upper 2D slice image. Alternatively theoverlap may be assessed based on a center of mass. That is, when acenter of mass of a connected component of the current 2D slice image issimilar to that of a connected component of the adjacent upper 2D sliceimage, the connected component of the current 2D slice image isassociated with the connected component of the adjacent upper 2D sliceimage. The center of mass may be calculated with an equal weight toevery pixel in a connected component as follows:

${C_{x} = {{\frac{\sum\limits_{i = 1}^{N}x_{i}}{N}\mspace{14mu} {and}\mspace{14mu} C_{y}} = \frac{\sum\limits_{i = 1}^{N}y_{i}}{N}}},$

where C_(x) and C_(y) are x-axis and y-axis coordinates of the center ofmass, respectively, x_(i) and y_(i) are coordinates of the i-th pixel ofa connected component, and N is the total number of pixels contained inthe connected component.

In another aspect, the connectivity criteria may be based on an arearatio. In particular, a ratio of an area of a non-overlapping portion ofa connected component of the current 2D slice image to an area of anoverlapping area of the connected component of the current slice may becompared with a first predetermined value. For example, the ratio may becomputed by dividing an area of an overlapping portion of a connectedcomponent of the adjacent upper 2D slice image by an area of anon-overlapping portion of the connected component of the adjacent upper2D slice image. When the ratio is less than the first predeterminedvalue, the connected component of the current 2D slice image and thecorresponding connected component of the adjacent upper 2D slice imageare associated.

Returning to FIG. 4, a fourth axial 2D slice image 425 a is taken alongthe axial direction where three connected components 426-428 aredetected. Using the connectivity criteria and filtering techniquesdescribed above, the connected component 426 is associated with theconnected component 421, the connected component 427 is associated withconnected component 422, and the connected component 428 is associatedwith the connected component 423. Since the connected components 422 and423 were previously filtered out as being tool larger and not given thelabel active object, the connected components 427 and 428 are alsofiltered out and not designated as active objects in FIG. 425 b. Theconnected component 426, however, is associated with the connectedcomponent 421, and is ultimately labeled as part of active object 412,as shown in the image 425 b.

Axial 2D slice image 430 a is the fifth 2D slice image from the top 2Dslice image 410 a. Again, three connected components 431-433 aredetected in the 2D slice image 430 a. Based on the connectivity criteriaand the filtering processes described above, the connected component 431is associated with the connected component 426, the connected component432 is associated with connected component 427, and the connectedcomponent 433 is associated with the connected component 428. As inimage 425 b, because the connected components 427 and 428 are too largerto be labeled as active objects, the connected components 432 and 433are likewise not associated with an active objects and are removed fromthe analysis. The connected component 431 however, is associated withthe connected component 426 is, which has previously been associatedwith active object 412 as shown in the image 430 b.

As shown in the 2D slice images 430 a and 430 b, the area of theconnected component 431 is small compared to the area of the connectedcomponent 426 in the 2D slice image 425 a, the connected component 421in the 2D slice image 420 a, the connected component 416 in the 2D sliceimage 415 a, and the connected component 411 in the 2D slice image 410a, all of which are associated with the active object 412. In at leastone embodiment because the ratio of the area of the connected component431 to the area of the connected component 426 of the 2D slice image 425a is below a threshold, the active object 412 including the connectedcomponents 411, 416, 421, 426, and 431 may be finalized, meaning thatthe active object 412 is closed. After the active object is finalized,no other connected components are associated with the active object.

When the active object 412 is finalized, the length of the active objectmay be calculated by multiplying the number of 2D slice imagescontaining the active object by the distance between adjacent 2D sliceimages. Based on the length of the active object, a determination ismade as to whether the active object is the trachea. In an aspect, ifthe length of the active object is greater than 70 millimeters (mm), theactive object is identified as the trachea. In another aspect, if thelength of the active object is greater than or equal to 30 mm and lessthan or equal to 70 mm, the active object is identified as the trachea.When the length of the active object is less than 30 mm, the activeobject is not identified as the trachea.

FIGS. 5A and 5B are flowcharts of a method 500 for automaticallydetecting a trachea in accordance with an embodiment of the presentdisclosure. The method 500 starts at step 505, in which a 3D model of apatient's lungs is generated. The 3D model may be based on CT scan imagedata obtained during a CT scan of the patient's chest and stored in theDICOM image format. In an aspect, the imaging modality may also beradiography, tomogram produced by a CAT scan, MRI, ultrasonography,contrast imaging, fluoroscopy, nuclear scans, and PET.

In step 510, 2D slice images may be generated from the 3D model. Thegenerated 2D slice images may be binarized 2D slice images includingonly include black and white pixels. The 2D slice images may begenerated along the axial direction. Alternatively, the 2D slice imagesare generated along a direction other than the axial direction. In anaspect, the 2D slice images are generated at an equal distance apart sothat a distance between any two 2D slice images may be easilycalculated. In another aspect, the 2D slice images may be generated atdifferent distances but may include distance information indicating howfar apart each 2D slice image is from the top 2D slice image.

In step 515, a connected component is identified in the 2D slice images.As noted above, connected components are enclosed regions in each image,with only one color pixels (e.g., black as shown in FIG. 4). Anyconnected component identified in the top 2D slice image is labeled asan active object in step 520. Active objects are considered ascandidates for the trachea. In step 525, a counter i is set to two, andthe next 2D slice image is examined.

FIG. 5B shows a flowchart for associating and labeling connectedcomponents as a part of the method 500 for automatically detecting atrachea. In step 526, a determination is made as to whether a connectedcomponent in the ith 2D slice image is associated with a connectedcomponent in the (i−1)th 2D slice image. In an aspect, a connectedcomponent in a current 2D slice image may be associated with a connectedcomponent in the previous 2D slice image based on a location of theconnected component of each of the current and previous 2D slice images.When the connected components overlap, they are associated with eachother. Otherwise, the connected components are not associated.

When a determination is made that a connected component in the current2D slice image (i.e., ith 2D slice image) is not associated with aconnected component in the previous 2D slice image (i.e., (i−1)th 2Dslice image), the connected component of the current 2D slice image islabeled as an active object in step 528. Step 570 (FIG. 5A) is thenperformed.

When a determination is made that a connected component in the current2D slice image is associated with a connected component in the previous2D slice image in step 526, another determination is made as to whetherthe connected component of the previous 2D slice image is an active instep 530. After the labeling process, step 570 of FIG. 5A follows.

In a case where the connected component of the previous 2D slice imageis labeled as an active object, an association parameter R is calculatedbetween the connected components of the current 2D slice image and theprevious 2D slice image in step 534. The association parameter is basedon connectivity criteria, which is used to determine whether twoconnected components of neighboring 2D slice images are closely related.

In an aspect, the association parameter is an area ratio, which is aratio of an area of a connected component of the current 2D slice imageto an area of the corresponding connected component of the previous 2Dslice image. In step 536, the association parameter is compared with twopredetermined values. In a case where the association parameter R isless than a first predetermined value P₁, the connected component, whichis labeled as the active object, of the previous 2D slice image isfinalized in step 538. This case occurs when the area of the connectedcomponent of the current 2D slice image decreases significantly or iscompletely missing. For example, since the lower end of the tracheabranches out, an image of the bottom of the trachea may show a connectedcomponent, an area of which is much smaller than a cross-sectional areaof the trachea. The significant decrease in the area of a connectedcomponent may indicate that the bottom of a trachea is reached.

When the association parameter R is greater than or equal to the firstpredetermined value P₁ but less than or equal to a second predeterminedvalue P₂, the connected component of the current 2D slice image islabeled as the active object in step 540. In this case, the connectedcomponent of the current 2D slice image is considered a continuation ofthe active object identified in the preceding 2D slice images (e.g., atrachea candidate).

When the association parameter R is greater than the secondpredetermined value P₂, the label of the connected component of theprevious 2D slice image may be removed such that it is not labeled as anactive object in step 542. This occurs when the area of the connectedcomponent of the current 2D slice image increases significantly. As aresult, the association parameter may reach 100%. In such instances asecond inquiry is made at step 541 as to whether the diameter of theconnected component of the current image slice is greater than apre-determined threshold, for example 30 mm for a man and 25 mm for awoman. Such a diameter of connected component would indicate thatconnected component cannot be the trachea. Thus, the connected componentof the previous 2D slice image is not considered as a trachea. In step544, the label of the connected component of the current 2D slice imageis also removed such that it is not labeled as an active object.

After steps 538, 540, and 544, a determination is made as to whetherthere is an unprocessed connected component in the current 2D sliceimage. When a determination is made that an unprocessed connectedcomponent exists, steps 526-546 are repeated until no additionalunprocessed connected components are found in the current 2D sliceimage. When a determination is made that there are no more unprocessedconnected components in the current 2D slice image, step 570 of FIG. 5Afollows.

Turning now to FIG. 5A, the counter i is increased by one in step 570.In step 575, the counter i is compared with the number of 2D sliceimages, N. When the counter i is less than or equal to the number of 2Dslice images, N, the method reiterates at step 526 illustrated in FIG.5A. Otherwise, all connected components in each 2D slice image areprocessed. In step 580, a length of the active object is calculated.

In steps 585 and 586, the length of the active object is compared with apredetermined range of values. At step 590, if the length of the activeobject is larger than the predetermined values, it is determined to bethe trachea. Similarly, at step 592, if the length of the active objectis within the predetermined range, it is labeled as potentially beingthe trachea, and a clinician may have to confirm this before the nextstep of the ENB procedure can be undertaken. At step 595, if the activeobjects are smaller than the predetermined range, automatic detection ofthe trachea fails, and manual identification and marking of the tracheais necessary. In an aspect, the predetermined range is 30 mm to 70 mm.Thus, if the length of an active object is more than 70 mm, it isdetermined to be the trachea, and if the length of an active object isbetween 30 mm and 70 mm, it is labeled as potentially being the trachea.In this way, the method 500 automatically detects a trachea from the 2Dslice images.

Returning now to FIG. 1, memory 120 includes application 122 such as EMNplanning and procedure software and other data that may be executed byprocessors 110. For example, the data may be the CT scan image datastored in the DICOM format and/or the 3D model generated based on the CTscan image data. Memory 120 may also store other related data, such asmedical records of the patient, prescriptions and/or a disease historyof the patient. Memory 120 may be one or more solid-state storagedevices, flash memory chips, mass storages, tape drives, or anycomputer-readable storage media which are connected to a processorthrough a storage controller and a communications bus. Computer readablestorage media include non-transitory, volatile and non-volatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer-readableinstructions, data structures, program modules or other data. Forexample, computer-readable storage media includes random access memory(RAM), read-only memory (ROM), erasable programmable read only memory(EPROM), electrically erasable programmable read only memory (EEPROM),flash memory or other solid state memory technology, CD-ROM, DVD orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store desired information and which can be accessed by device100.

Display 130 may be touch-sensitive and/or voice-activated, enablingdisplay 130 to serve as both an input device and an output device.

Graphics processors 140 may be specialized graphics processors whichperform image-processing functions, such as processing the CT scan imagedata to generate the 3D model, and process the 3D model to generate the2D slice images of the 3D model in the various orientations as describedabove, as well as the 3D renderings of the 3D model. Graphics processors140 may further be configured to generate a graphical user interface(GUI) to be displayed on display 130. The GUI may include views showingthe 2D image slices, the 3D rendering, among other things. Inembodiments, graphics processors 140 may be specialized graphicsprocessors, such as a dedicated graphics processing unit (GPU), whichperforms only the image processing functions so that the one or moregeneral processors 110 may be available for other functions. Thespecialized GPU may be a stand-alone dedicated graphics card, or anintegrated graphics card.

Network interface 150 enables device 100 to communicate with otherdevices through a wired and/or wireless network connection. In anembodiment, device 100 may receive the CT scan image data from animaging device via a network connection. In other embodiments, device100 may receive the CT scan image data via a storage device, such as adisk or other external storage media known to those skilled in the art.

Input interface 160 is used for inputting data or control information,such as setting values, text information, and/or controlling device 100.Input interface 160 may include a keyboard, mouse, touch sensor, camera,microphone, or other data input devices or sensors used for userinteraction known to those skilled in the art.

Further aspects of image and data generation, management, andmanipulation useable in either the planning or navigation phases of anENB procedure are more fully described in commonly-owned U.S.Provisional Patent Application Ser. No. 62/020,220 entitled “Real-TimeAutomatic Registration Feedback”, filed on Jul. 2, 2014, by Brown etal.; U.S. Provisional Patent Application Ser. No. 62/020,177 entitled“Methods for Marking Biopsy Location”, filed on Jul. 2, 2014, by Brown.;U.S. Provisional Patent Application Ser. No. 62/020,240 entitled “Systemand Method for Navigating Within the Lung”, filed on Jul. 2, 2014, byBrown et al.; U.S. Provisional Patent Application Ser. No. 62/020,238entitled “Intelligent Display”, filed on Jul. 2, 2014, by Kehat et al.;U.S. Provisional Patent Application Ser. No. 62/020,242 entitled“Unified Coordinate System for Multiple CT Scans of Patient Lungs”,filed on Jul. 2, 2014, by Greenburg.; U.S. Provisional PatentApplication Ser. No. 62/020,245 entitled “Alignment CT”, filed on Jul.2, 2014, by Klein et al.; U.S. Provisional Patent Application Ser. No.62/020,250 entitled “Algorithm for Fluoroscopic Pose Estimation”, filedon Jul. 2, 2014, by Merlet.; U.S. Provisional Patent Application Ser.No. 62/020,261 entitled “System and Method for Segmentation of Lung”,filed on Jul. 2, 2014, by Markov et al.; U.S. Provisional PatentApplication Ser. No. 62/020,258 entitled “Cone View—A Method ofProviding Distance and Orientation Feedback While Navigating in 3D”,filed on Jul. 2, 2014, by Lachmanovich et al.; and U.S. ProvisionalPatent Application Ser. No. 62/020,262 entitled “Dynamic 3D Lung MapView for Tool Navigation Inside the Lung”, filed on Jul. 2, 2014, byWeingarten et al., the entire contents of all of which are herebyincorporated by reference.

Although embodiments have been described in detail with reference to theaccompanying drawings for the purpose of illustration and description,it is to be understood that the inventive processes and apparatus arenot to be construed as limited thereby. It will be apparent to those ofordinary skill in the art that various modifications to the foregoingembodiments may be made without departing from the scope of thedisclosure.

1-18. (canceled)
 19. A non-transitory computer-readable storage mediumstoring instructions for detecting a trachea of a patient, theinstructions, when executed by one or more processors, cause a computerto: label a connected component in a top slice image of the plurality ofslice images as an active object, the plurality of slice images beingalong an axial direction from a three-dimensional (3D) model of apatient's chest; associate the connected component in a current sliceimage of the plurality of slice images with a corresponding connectedcomponent in a previous slice image of the plurality of slice imagesbased on a connectivity criterion; and label each connected component inthe current slice image associated with a connected component in theprevious slice image as the active object.
 20. The non-transitorycomputer-readable storage medium according to claim 19, wherein theinstructions, when executed by the one or more processors, further causethe computer to: obtain the three-dimensional (3D) model of a patient'schest; generate the plurality of slice images of the 3D model along theaxial direction; identify the connected component in each of theplurality of slice images; and identify the active object as thetrachea, based on a length of the active object.
 21. The non-transitorycomputer-readable storage medium according to claim 20, wherein the 3Dmodel is generated based on image data obtained by an imaging deviceusing a tomographic technique, radiography, a tomogram produced by acomputerized axial tomography scan, magnetic resonance imaging,ultrasonography, contrast imaging, fluoroscopy, nuclear scans, orpositron emission tomography.
 22. The non-transitory computer-readablestorage medium according to claim 20, wherein the instructions, whenexecuted by the one or more processors, further cause the computer tofinalize the active object in the previous slice image.
 23. Thenon-transitory computer-readable storage medium according to claim 22,wherein the plurality of slice images are spaced at an equal distanceapart from each other.
 24. The non-transitory computer-readable storagemedium according to claim 23, wherein the instructions, when executed bythe one or more processors, further cause the computer to calculate alength of a finalized active object by multiplying a number of theplurality of slice images including the finalized active object minusone and the distance between each slice image.
 25. The non-transitorycomputer-readable storage medium according to claim 24, wherein, whenthe length of the finalized active object is greater than or equal to 70mm, the method further comprises indicating that the trachea isidentified.
 26. The non-transitory computer-readable storage mediumaccording to claim 24, when the length of the finalized active object isgreater than or equal to 30 mm but less than 70 mm, the method furthercomprises indicating that the trachea is potentially identified.
 27. Thenon-transitory computer-readable storage medium according to claim 24,wherein, when the length of the finalized active object is less than 30mm, the method further comprises indicating that the trachea has notbeen identified.
 28. The non-transitory computer-readable storage mediumaccording to claim 22, wherein the finalizing of the active object inthe previous slice image is based on an association parameter.
 29. Thenon-transitory computer-readable storage medium according to claim 28,wherein the association parameter is an area ratio calculated bydividing an area of the connected component in the current slice imageby an area of the corresponding connected component in the previousslice image.
 30. The non-transitory computer-readable storage mediumaccording to claim 28, wherein the association parameter is a ratiobetween a number of coordinates of the connected component in thecurrent slice image, which match coordinates of the corresponding activeobject in the previous slice image, and a number of non-matchingcoordinates of the connected component in the current slice image. 31.The non-transitory computer-readable storage medium according to claim28, wherein the association parameter is an area of the connectedcomponent in the current slice image.
 32. The non-transitorycomputer-readable storage medium according to claim 28, wherein theinstructions, when executed by the one or more processors, further causethe computer to remove the label of the corresponding active object inthe previous slice image as an active object when the associationparameter is greater than a predetermined value.
 33. The non-transitorycomputer-readable storage medium according to claim 32, wherein theinstructions, when executed by the one or more processors, further causethe computer to remove the label of the connected component in thecurrent slice image as an active object when the association parameteris greater than the predetermined value.
 34. The non-transitorycomputer-readable storage medium according to claim 28, wherein anactive object is finalized when the association parameter is less than apredetermined value.
 35. The non-transitory computer-readable storagemedium according to claim 28, wherein the instructions, when executed bythe one or more processors, further cause the computer to label theconnected component in the current slice image as the active object whenthe association parameter is greater than or equal to a firstpredetermined value and less than or equal to a second predeterminedvalue.
 36. The non-transitory computer-readable storage medium accordingto claim 20, wherein a connected component in the current slice image isassociated with the corresponding connected component in the previousslice image when coordinates of a pixel in the connected component inthe current slice image matches coordinates of a pixel in thecorresponding connected component in the previous slice image.
 37. Thenon-transitory computer-readable storage medium according to claim 20,wherein a connected component in the current slice image is associatedwith the corresponding connected component in the previous slice imagewhen a difference between a center of mass of the connected component inthe current slice image and a center of mass of the correspondingconnected component in the previous slice image is less than apredetermined value.
 38. The non-transitory computer-readable storagemedium according to claim 20, wherein a connected component in thecurrent slice image is associated with a corresponding connectedcomponent in the previous slice image when a difference between an areaof the connected component in the current slice image and an area of thecorresponding connected component in the previous slice image is lessthan a predetermined value.